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ABSTRACT 

Comprehensive evaluations of two dropout 
prevention/reentry demonstration programs, sponsored by the 
Department of Education, one funded under the School Dropout 
Demonstration Assistance Program (SDDAP) and one funded under the 
Cooperative Demonstration Program (CDP) , are provided in this paper. 
Both programs were designed to demonstrate effective strategies to 
encourage at-risk students to complete school or to reenter a high 
school equivalency program. Evaluation goals were to identify the 
extent of implementation success and implementation factors for 
success. The first part of the paper offers a brief descriptive 
profile of 25 programs, 15 SDDAP and 10 CDP. The evaluation design, 
which obtained data through document analysis, interviews, 
observation, and focus groups is described next. Findings indicate 
that program implementation can be enhanced by the following 
strategies: expanding existing programs rather than initiating new 
ones; correlating appropriate resource allocation with the particular 
implementation model; considering the local environment; and 
including a planning period in the implementation timeline. Two 
figures are included. (LMI) 
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DROPOUT PREVENTION DEMONSTRATION PROJECTS: 



FACTORS THAT AFFECT IMPLEMENTATION AND EFFECTIVENESS 1 



Becky Jon Haywaid 
Research Triangle Institute 



The U.S. Department of Education (ED) is currently sponsoring comprehensive 
evaluations of two dropout prevention/reentry demonstration programs, one funded under the 
School Dropout Demonstration Assistance Program (SDDAP) in the Office of Elementary and 
Secondary Education (OESE), and the other funded under the Cooperative Demonstration 
Program (CDP) in the Office of Vocational and Adult Education (OVAE). Both programs 
intend to demonstrate effective strategies for encouraging at-risk children and youth to remain 
in and complete school or, for youth who have already dropped out of school, to reenter an 
educational program leading to a high school diploma or GED. Under SDDAP, grantees 
received funding for demonstration projects for a period of two years beginning in school 
year 1988-89; subsequently, ED added a third year of funding, with projects scheduled to end 
in spring 1991. Under CDP, grantees received funding for a three-year period, beginning in 
School Year 1989 and ending in spring 1992. 

Organizations awarded grants to implement dropout prevention/reentry projects under 
these programs included school districts or consortia of districts, state education agencies, 
universities, community-based organizations, and a variety of other education organizations 
such as child service demonstration centers or intermediate service units. Under the 
specifications of the grant announcement for both programs, grantees were authorized either 
to replicate an existing dropout prevention/reentry model that had been tested and proven 
effective in other settings, expand an existing model that the grantee had already designed and 
implemented with other funds, or (for SDDAP projects) establish a new model of services 
based on analysis of the specific needs of at-risk children and youth in the community. 

ED has funded two comprehensive program evaluations whose purpose is to assess the 
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effectiveness of projects funded under these two demonstration programs in encouraging 
participants to remain in and complete (or return to and complete) their high school 
education, A key component of each evaluation is a detailed implementation study, whose 
purpose is to analyze factors that are conducive to successful project implementation and to 
identify those components of projects that appear to have positive effects on participant 
outcomes. The implementation analysis conducted for the SDDAP evaluation was cross- 
sectional in nature, with projects selected for in-depth study visited once, in spring 1990- The 
CDP evaluation is conducting a longitudinal implementation evaluation, with projects visited 
five times over a three-year period. 

This paper provides preliminary findings from these two implementation components 
of the comprehensive evaluations. Key questions addressed include: 

• To what extent have projects succeeded in implementing the project models 
they intended to implement? 

• What implementation factors can be identified that appear to facilitate or deter 
improved participant outcomes? 

To provide a context for our findings, we provide (1) a brief descriptive profile of the 25 
projects included in the implementation analyses (15 in the SDDAP evaluation and 10 in the 
CDP evaluation), (2) an overview of the design of the two implementation studies, and (3) a 
discussion of preliminary findings. 

Profile of the Implementation Sites 

SDDAP 

The SDDAP evaluation selected 15 of the 89 projects for "intensive* 1 evaluation, 
including assessment of project implementation. Exhibit 1 provides an overview of the key 
characteristics of these projects. Six of the projects serve children in grades K - 5, while 
eight serve high school youth and one serves reentering dropouts who are either making up 
credits for graduation or working toward a GED. Two of the projects serve first graders 
exclusively: one has implemented an after-school whole language reading program, while the 
other has placed college students in the classroom to work individually with first graders to 
help them feel more comfortable about school. 
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Exhibit 2-1 



Characteristics of SDDAP and CDP Projects 
in the In-Depth Studies 







Grade levels served 




K-5 


6 


6-8 


8 


y - iz 




Reentry 


L 


Purpose of the grant 




Replicate a validated model 


9 


Expand an existing project 


9 


Establish a new project 


9 


Organizational structure 




After school/Saturday/extended day 


4 


Pull-out 


6 


Ouster/enrichment 


3 


Alternative school 


10 


School-within-a-school 


3 


Schoolwide protect 


1 


Key services 




Academic remediation/GED prep 


18 


Counseling 


12 


Academic enrichment/innovative instruct. 


7 


Vocational education 


6 


Other employability-ielated activities 


10 


Parent component 


4 


Proactive attendance monitoring 


4 



3 



ERIC 



4 



Eight of the IS projects were awarded grants to expand projects already in place. In 
terms of organizational structure, four serve participants outside the school day, three are pull- 
out, three have implemented a cluster or enrichment design, and three are alternative high 
schools. Twelve of the projects emphasize academic remediation (or GED preparation) as a 
key service, while eight provide group or individual counseling, and four have implemented 
academic enrichment activities. Four provide employability related activities-in two of these 
projects, students can "earn" part-time jobs as a reward for improved attendance and academic 
achievement. 

Finally, as shown in Exhibit 1, most of the projects have articulated specific goals that 
supplement or expand the primary goal of dropout prevention. As might be expected most 
(13 in all) emphasize improvement in academic achievement. Others emphasize improvement 
in participants 9 self-esteem (one projects has this objective as its sole emphasis), increased 
parental involvement in their children's educational programs, and use of jobs as an incentive 
to increase participants' affiliation with school. Only two projects, however, have undertaken 
proactive strategies for improving attendance, although most projects do list improvement in 
attendance as a key project goal. 
CDP 

Because the demonstration projects funded under CDP arc required to include a 
vocational component, they target services to high school students, where secondary 
vocational education is provided (see Exhibit 2). As noted, one of the projects targets 
services to youth who have already dropped out of high school. Nine of the projects intended 
to replicate an existing, tested model. Two grantees planned to replicate Project Coffee in a 
total of seven local sites. Coffee is an alternative school with an abbreviated school day that 
was originally developed to serve youth identified as seriously emotionally disturbed (SED) 
who were classified as special education students under the Individuals With Disabilities 
Education Act (IDEA). One grantee intended to replicate the Peninsula Academies model by 
establishing Business Technology Academies in two high schools within one high school 
district. Two grantees planned to adapt a locally developed project model (one of which has 
been validated by the National Diffusion Network for special education students), while the 
final grantee in the evaluation is expanding existing services to at-risk youth. 
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Grade levels served 






9- 12 


a 




Reentry 


i 
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Kepncate a validated model 


9 




Adapt a locally developed project 


2 




Expand an existing project 


1 




Organizational structure 






Alternative school 


7 




School-within-a-school 


2 
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JC ey services 






Vocational education 


6 




Career awareness/employability devt 


6 




Paid work experience 


6 




Academic remediation 


6 




Academic enrichment 


3 




Counseling 


4 




Incentives 


8 




Key ancillary goals 






Improvement in attendance (proactive) 


2 




Improvement in self-esteem 


4 




Improvement in self-management 


3 




Realistic post-school expectations 


4 





^ome of the projects participating in the evaluation axe located in multiple sites. Thus, the number of 
individual sites is 12 rather than 10. 
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Consistent with these models, seven of our evaluation sites are in some sense 
"alternative" schools, two are schools-within-a-school, and three operate in the "regular" 
education setting, providing supplemental services to participants. 

Half of the 12 project sites provide standard vocational education programs (i.e., a 
sequenced curriculum of occupational preparation in a specific area, such as welding, auto 
mechanics, or secretarial preparation). Six provide career awareness or employability 
development services rather than standard vocational education, and six have arrangements in 
place for participants to work at paid jobs, although only three sites initiated this component 
in the first year of their grants. 

Six projects provide academic remediation (and one provides remedial assistance to 
participants in their vocational courses), and three have implemented academic enrichment 
components. Of the four sites that provide some form of counseling, one has implemented a 
well -organized group counseling component for all participants; the others offer more 
traditional "guidance" services. Finally, the four sites operating under a grant to a state board 
of vocational education provide monetary incentives for attendance. Project participants earn 
$1 per day for school attendance and a $5 monthly bonus if they miss no school during the 
month. Other projects offer more traditional incentives, such as field trips or pizza parties. 

As with the SDDAP projects, the CDP demonstrations have developed "intermediate" 
goals to support the overall goal to reduce dropping out among at-risk youth. These range 
from activities to improve self-esteem (four projects) or self-management skills (three 
projects), to strategies to assist participants in developing realistic expectations and plans for 
their futures (four projects). Only two of the project sites have implemented proactive 
attendance monitoring components. 

Overview of the Implementation Designs 
In general, the designs for the implementation assessment components of the two 
evaluations are similar. Prior to field work, we analyzed available documentation for each of 
the selected projects to determine their principal characteristics along several dimensions, 
including (1) organizational location and context, (2) configuration of intended project 
components and services, (3) staffing and resources, (4) planning and schedule for 
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implementation, (5) strategies for identifying and recruiting participants, (6) staff training and 
support, and (7) linkages and coordination with other organizations, such as the private sector, 
other service providers, and the like. For the CDP projects, we developed an instrument that 
would permit us to measure the amount and quality of project implementation on these 
dimensions over time. For the SDDAP projects, which we were to visit only once, we 
omitted this component, whose purpose was, in part, to assess the projects' fidelity to their 
intended model over time and obtain detailed information on the nature and causes of 
adaptations. Since the SDDAP projects had already been operating for nearly two years, we 
were unable to capture adaptations they had made over time; thus we focused on what was in 
place at the time of the field work. We did, however, analyze the status of implementation in 
the context of the projects' services. For both studies, we obtained information on the 
configuration of services and on the relative "intensity" of those services in order to be able 
to explain participant outcomes. 

Our field work design involved several different types of activities: (1) personal 
interviews with relevant administrative staff of the grantee, the project director, project 
teachers, counselors, and other staff as appropriate, and representatives of "linkage" 
organizations; (2) collection of information on school and community context, project 
monetary and other resources, staff training and experience prior to and following assignment 
to the project, and other information available about each project (e.g., annual self-evaluation 
reports); (3) detailed documentation of the "treatment" at the participant level; (4) classroom 
or service observations; and (5) focus groups with participants (for SDDAP projects only). 

Preliminary Findings on Project Implementation 

In this section we present some of our early findings on factors that appear to facilitate 

or impede project implementation across the 25 projects included in the two evaluations and 

describe some of the components of these projects that appear to be effective in encouraging 

participants to remain in— and affiliate with-schooL 

• To what extent have projects succeeded in implementing the project models 
they intended to implement? 

As might be expected, in general the grantees whose plan was to expand an existing 



project or adapt a locally developed model experienced greater early success in getting their 
projects started than did those whose intent was to replicate a validated model. Expansion of 
existing projects generally did not require identifying and hiring a new project director, 
locating new facilities, or major additions in equipment. Rather, these projects typically 
increased their service capacity by adding staff and were able to increase the numbers of 
participants they were able to serve, with minimal additional need for other changes. 
Similarly, several grantees either transported or adapted models with which they had 
extensive experience. For example, the grantee that replicated the Peninsula Academies 
model was the school district that first implemented this model. That district had in place 
two Academy programs that have been operating successfully for a number of years, and they 
initiated new Academies in two other high schools in the district. Thus, while they did need 
to arrange space, designate lead and other teachers, recruit students and private sector 
partners, the expertise for accomplishing these tasks was already in place. As another 
illustration, the grantee that adapted the locally developed special education model to meet the 
needs of at-risk students only needed to develop appropriate agreements with an area 
vocational-technical center that is housing the project and hire additional staff. In all of these 
instances, the requisite knowledge and experience were available, and project administrators 
had relatively little difficulty in getting underway. 

Conversely, some grantees experienced considerable difficulty in replicating models 
about which they had litde prior knowledge. Two grantees (a state board of vocational 
education and a university) won awards to replicate Project Coffee, in a total of seven local 
sites. Only one of the sites has implemented a project that would be recognizable to the 
originators of this model. A number of factors help to account both for the "success" and the 
early "failures." On the positive side, the successful replication was able (1) to attract a 
sizable amount of state funding to supplement the small CDP grant; (2) to obtain high-level 
district support for the project, which resulted in relaxation of state-mandated attendance 
requirements, modification of fairly restrictive student behavioral standards, and location of a 
facility to house the alternative school; (3) to attract a highly qualified and committed 
individual as project director, who subsequently hand picked staff based primarily on their 
experience with and commitment to serving at-risk youth. 
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The six sites that have experienced difficulty in implementing the model also offer 
some lessons about factors that might be anticipated-and perhaps averted-during the early 
stages of project operations. First, projects need to select models that match available fiscal 
resources. One project in particular has been unable to attract local funds to supplement the 
grant and has been unable to implement a model that is relatively expensive to operate. 
Another project generally has adequate resources but has been located in a vocational- 
technical school whose director has been insistent on maintaining close control over all 
activities in his school. He has been unwilling to make changes needed to accommodate the 
project model in such areas as adjustments in scheduling, provision of inservice support to 
teachers, or recognizing the authority of the project director to make and implement decisions 
regarding the project. Four of the projects, with the project director located in the state office 
and the sites widely scattered around the state, were unable to obtain status as a pilot project. 
As a consequence, local instructors hired for the projects could not obtain waivers necessary 
to provide instruction for which students could obtain credit toward graduation, and the 
concept of an alternative school had to be abandoned. Three of the projects revised their 
plans by offering pull-out tutorial services to students and some individualized career 
preparation/employability instruction. The fourth implemented a GED preparation program 
for dropouts. Further, owing in part to severe economic problems in the projects* locations, 
none was able to establish an entrepreneurial business, and only two were located in areas 
where students had access to any vocational education at all. Further, this problem has meant 
that participants have not been able to participate in work experience-there are no jobs 
available. 

These illustrations suggest several lessons for localities intending to implement dropout 
prevention/reentry projects. First, in general it may be more feasible to expand existing 
projects or to implement a model with which the grantee has experience. Replication of one 
of the validated models, in the absence of experience with that model, will likely require an 
investment in capacity building and possibly in. ongoing support for staff responsible for the 
project. Second, grantees need to have a good sense of the resources required, such that 
available resources match the requirements of the service configuration they intend to 
implement. Third, caiefiil consideration of the local setting of the project is essential. For 
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example, planning for work experience as a major project component may be infeasible in 

locations with severe unemployment. Similarly, if state or district waivers are required for 

some project activities, formal agreements may be required as grant proposals are developed. 3 

• What project components appear to facilitate or deter improved participant 
outcomes? 

One factor that had a strong effect on project implementation in the early phases of 
the demonstrations has to do with timing of die grant Owing in part to the timing of the 
grant competitions, grantees had little time to engage in planning after they were notified of 
awards. In most instances, applicants are unable to undertake planning and implementation 
activities until award. Since both competitions awarded grants in the summer, after staff 
assignments for the upcoming school year had been made, a number of the grantees did not 
have a project director or a specified organizational location for the project. (In many 
instances, particularly in large urban school districts, the authors of the grant were located in 
a grant-writing division and were not the persons who would actually staff the project.) For 
some grantees, therefore, initiation of project activities was significantly delayed, in one 
instance for more than a year, after grant award. One implication of this factor is that often 
the individuals who end up staffing the project may have little knowledge of the project 
design and have to hire staff, learn the model, work out operational relationships with 
participating schools, recruit and admit students, and solve myriad logistical problems in a 
very short period. The projects we visited that were successful in their early phases avoided 
this difficulties in one of two ways: either they factored in a planning period, often the first 
semester of the grant, or they were able to dedicate a project director on the chance that they 
would be awarded a grant. Since the latter is not feasible in many localities, inclusion of a 
planning period in the schedule appears to be critical in effective project implementation. 

In some instances, apparently minor aspects of project implementation can have an 
unexpected effect on project success. For example, in one district with two project sites, one 



*We recognize that prior planning, however useful, is not always possible. One project, whose service model 
relies on field trips to local area businesses as part of the career development component, bad to drop this 
activity when the district changed its policies regarding transporting students during school hours. Participants 
perceived that something had been taken away from them, and the resulting morale problems were an issue for 
project staff. 
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of the components planned for the projects was proactive attendance monitoring, and an 
individual was hired to check on students on a daily basis, including visiting classes, 
counseling students, and telephoning and visiting parents. While this person was responsible 
for both sites, she was located at one of the sites and resides in the community in which most 
of that site's participants live. Predictably, most of her efforts have focused on that school, 
and the study's outcome analyses have shown significant improvement in attendance for those 
participants. Also predictably, the other site has not shown similar improvement; further, 
staff at the second site have expressed displeasure over their lack of access to this support 
service. 

Finally, as implementation research has consistendy demonstrated, capability, 
creativity, and commitment on the part of project staff are key to project success. To 
illustrate, the director of a project located in a community characterized by severe economic 
and employment problems has devised a particularly creative solution to the project's planned 
work experience component. She has developed a culturally sensitive career development 
curriculum and has arranged for her reentering participants to gain work experience by 
working for a local foundation that has a grant to record oral history from elders in the 
community. This strategy has significantly improved participants' attendance, self-esteem, 
and sense of prospects for the future. 



Plans for Continuing Investigation 
The final report on the implementation component for the SDDAP evaluation is 
currently under review, but the CDP longitudinal implementation study will continue to assess 
project implementation, and its influence on participant outcomes, for another 18 months. 
The woik we are conducting will continue to investigate changes over time in factors that 
affect project implementation, and our analyses will focus on associations between the level 
and quality of implementation and the effects that these projects achieve in terms of 
improvements in participants' educational experiences and outcomes. 
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